Image sensing operator input device

ABSTRACT

A computer input device detects images on a surface. The computer input device generates input information indicative of a change event when the device switches from reading one predetermined pattern to reading another predetermined pattern. The present invention can also be directed to a method of using the input device or printable medium with the predetermined patterns disposed thereon.

REFERENCE TO CO-PENDING APPLICATION

Reference is made to co-pending U.S. patent application Ser. No.09/036,809, filed Mar. 9, 1998, entitled “OPERATOR INPUT DEVICE”, nowU.S. Pat. No. 6,172,354 and assigned to the same assignee as the presentapplication.

INCORPORATION BY REFERENCE

The following U.S. patent is hereby fully incorporated by reference:

U.S. Pat. No. 5,581,094 issued to Hara et al., entitled “PHOTODETECTORARRAY COMPRISING PHOTO DETECTORS, AND OBJECT DETECTOR COMPRISING THEPHOTO DETECTOR ARRAY AND AN OBJECT DETECTING PROCEDURE”, and assigned toMitsubishi Electric Corporation.

BACKGROUND OF THE INVENTION

The present invention relates to an input device for a computer system.More specifically, the present invention relates to an input device forproviding position information to the computer system based on movementof the input device.

A traditional computer input device, such as a mouse, includes ahousing, with a ball mounted in the housing. The ball is eitherconfigured in a traditional manner in which, in the normal workposition, the ball engages a work surface and rotates based on theuser's movement of the mouse across the work surface. The ball may alsobe provided as a track ball, which is rotated by digital manipulationfrom the user. In either case, position encoders are used to detectrotation of the ball in the mouse, and to provide position informationindicative of that rotation to the computer. In many instances, theposition information is used to control movement of a visual image (suchas a mouse cursor) on the display screen of the computer.

Also, in one prior device, a mouse is configured with the track ballarrangement described above. The track ball is preprinted with a uniformpredetermined image. A charge coupled device is used to detect the imageon the track ball and detect movement of the image. Movement of thepredefined image is used to provide position information to thecomputer.

However, the prior computer mouse which uses the charge coupled deviceconfiguration has a number of significant disadvantages. First, thereaction time of charge coupled devices is quite slow. In addition,processing an image signal from a charge coupled device iscomputationally intensive and takes a relatively large, and expensiveprocessor. Also, charge coupled devices are highly sensitive tosaturation. In other words, if the ambient light conditions arevariable, charge coupled devices do not perform well. In addition, if anextraneous light source, such as a relatively bright light, is directedtoward the image producing surface, the charge coupled devices caneasily become saturated and their performance then quickly degrades.

Further, another prior computer mouse commercially available from MouseSystems of CA included a mouse with an LED which was used in conjunctionwith a mouse pad having a predetermined, uniform pattern thereon. Thepattern was formed by a uniform grid of blue and red lines. Theemissions from the LED was reflected off of the mouse pad to a detectorwhich provided an analog output signal. The signal was in the form of awaveshape with peaks corresponding to the different colored grid lines.From this waveform, the lines were counted and interpolated to obtainposition information. Such a mouse system requires a mouse pad with aspecial uniform pattern implemented thereon.

In addition, typical mouse pointing devices are operated using a fixedscale and resolution. A fixed scale means that the mouse must always bemoved a given distance over a work surface in order to move the cursorimage on the computer screen a given number of pixels. For example, inorder to move the cursor by 200 pixels on a computer screen having aresolution of 200 dots per inch (dpi), the mouse must be moved a fixeddistance over the work surface, such as two inches. Each time the mouseis moved two inches, no matter what surface it is moved over, thatmovement will always correspond to movement of the cursor image by 200pixels.

Fixed resolution refers to the resolution of the computer screen forwhich the smallest detectable discrete movement of the mouse will changethe cursor position on the computer screen by only a single pixel. Forexample, some conventional mice have a resolution generally in the rangeof 200-400 dpi. This means that the smallest discrete movement of themouse which is detectable by the position encoding mechanism in themouse will change the cursor position on the display screen by only asingle pixel for screens having a resolution in the range of 200-400dpi. However, if the computer screen has a higher resolution, such as1200 dpi, the smallest detectable discrete movement of the mouse maycause the cursor image to move 4-6 pixels on the display screen. Somecurrent mice, with highly accurate position encoding mechanisms, canachieve a resolution of 1200 dpi.

As stated above, the scale and resolution of conventional mouse pointingdevices are fixed. In order to change the scale or resolution, the useris typically required to load a different mouse driver which modifiesthe behavior of the mouse to change the resolution or to change thescale. Alternatively, a user can also use a separate control panelapplet to trigger the software device driver to change the scale andresolution. Using a control applet to change scale or resolution can bequite cumbersome.

In some applications, items to be selected on the screen are quitesmall. Therefore, it can be difficult to select items when a highresolution monitor is used. In some instances, a single pixel offset canchange the selection from one targeted item (or option) to another. Onemethod which can be used to overcome this difficulty in suchapplications is to maintain the resolution and magnification of thecomputer screen at its nominal level and to decrease the resolution ofthe pointing device. However, as indicated above, conventional methodsfor changing the resolution of the pointing device can be quitecumbersome, particularly when a user desires to change the resolution anumber of times while operating within a single application.

SUMMARY OF THE INVENTION

A computer input device detects images on a surface. The computer inputdevice generates input information indicative of a change event when thedevice switches from reading one predetermined pattern to readinganother predetermined pattern. The present invention can also bedirected to a method of using the input device or printable medium withthe predetermined patterns disposed thereon.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an exemplary environment for implementingan input device in accordance with the present invention.

FIG. 2A is a functional block diagram of a computer and an inputpointing device as used in one embodiment of the present invention.

FIG. 2B illustrates a packet of information generated by an inputpointing device for transmission to the computer.

FIG. 3 illustrates a computer input device, shown in partial sectionaland partial block diagram form, in accordance with one embodiment of thepresent invention.

FIG. 4 is a block diagram illustrating use of the pointing device shownin FIG. 3 in conjunction with a mouse pad or work surface having apredetermined pattern thereon.

FIGS. 5A and 5B illustrate operation of the input device in recognizingthe predetermined pattern shown in FIG. 4.

FIG. 6 illustrates a mouse pad or work surface having a variety ofpredetermined patterns disposed thereon for providing an input devicewith a variable resolution.

FIG. 7 is a view of a mouse pad or work surface having predeterminedcoded images disposed thereon.

FIG. 8 is a diagram which illustrates projection of the predeterminedcoded images on an X-Y coordinate system.

FIG. 9 illustrates a packet of information generated by an input devicein accordance with one embodiment of the present invention.

FIGS. 10A and 10B represent a flow diagram illustrating operation of aninput device in accordance with one embodiment of the present inventionusing the data packet illustrated in FIG. 9.

FIG. 11 his a block diagram illustrating the use of an input device toprovide specialized messages to the computer in accordance with anotherembodiment of the present invention.

FIGS. 12A and 12B represent a flow diagram illustrating the operation ofthe input device shown in FIG. 11.

FIG. 13 is a flow diagram illustrating the operation of an applicationin conjunction with an input device in accordance with anotherembodiment of the present invention in which the application generates asheet having application specific images disposed thereon.

FIG. 14 illustrates another mouse pad or work surface having a varietyof patterns or images disposed thereon.

FIG. 15A is a functional block diagram illustrating the use of an inputdevice in a learn mode in accordance with another embodiment of thepresent invention.

FIG. 15B illustrates one illustrative embodiment of an orientationcompensating code.

FIG. 16 is a flow diagram illustrating operation of an input device andcomputer in a learn mode in accordance with one embodiment of thepresent invention.

DETAILED DESCRIPTION OF THE ILLUSTRATIVE EMBODIMENTS

The present invention provides a user input device for generatingposition information and providing that information to a computersystem. The position information is generated based on detected movementof the user input device, or a portion thereof. The movement is detectedby identifying a pattern or image on a surface movable relative to theuser input device and monitoring relative movement of the pattern.

OVERVIEW

FIG. 1 and the related discussion are intended to provide a brief,general description of a suitable computing environment in which theinvention may be implemented. Although not required, the invention willbe described, at least in part, in the general context ofcomputer-executable instructions, such as program modules, beingexecuted by a personal computer or other computing device. Generally,program modules include routine programs, objects, components, datastructures, etc. that perform particular tasks or implement particularabstract data types. Moreover, those skilled in the art will appreciatethat the invention may be practiced with other computer systemconfigurations, including hand-held devices, multiprocessor systems,microprocessor-based or programmable consumer electronics, network PCs,minicomputers, mainframe computers, and the like. The invention is alsoapplicable in distributed computing environments where tasks areperformed by remote processing devices that are linked through acommunications network. In a distributed computing environment, programmodules may be located in both local and remote memory storage devices.

With reference to FIG. 1, an exemplary environment for the inventionincludes a general purpose computing device in the form of aconventional personal computer 20, including processing unit 21, asystem memory 22, and a system bus 23 that couples various systemcomponents including the system memory to the processing unit 21. Thesystem bus 23 may be any of several types of bus structures including amemory bus or memory controller, a peripheral bus, and a local bus usingany of a variety of bus architectures. The system memory includes readonly memory (ROM) 24 a random access memory (RAM) 25. A basicinput/output 26 (BIOS), containing the basic routine that helps totransfer information between elements within the personal computer 20,such as during start-up, is stored in ROM 24. The personal computer 20further includes a hard disk drive 27 for reading from and writing to ahard disk (not shown), a magnetic disk drive 28 for reading from orwriting to removable magnetic disk 29, and an optical disk drive 30 forreading from or writing to a removable optical disk 31 such as a CD ROMor other optical media. The hard disk drive 27, magnetic disk drive 28,and optical disk drive 30 are connected to the system bus 23 by a harddisk drive interface 32, magnetic disk drive interface 33, and anoptical drive interface 34, respectively. The drives and the associatedcomputer-readable media provide nonvolatile storage of computer readableinstructions, data structures, program modules and other data for thepersonal computer 20.

Although the exemplary environment described herein employs a hard disk,a removable magnetic disk 29 and a removable optical disk 31, it shouldbe appreciated by those skilled in the art that other types of computerreadable media which can store data that is accessible by a computer,such as magnetic cassettes, flash memory cards, digital video disks,Bernoulli cartridges, random access memories (RAMs), read only memory(ROM), and the like, may also be used in the exemplary operatingenvironment.

A number of program modules may be stored on the hard disk, magneticdisk 29, optical disk 31, ROM 24 or RAM 25, including an operatingsystem 35, one or more application programs 36, other program modules37, and program data 38. A user may enter commands and information intothe personal computer 20 through input devices such as a keyboard 42 andpointing device 40. Other input devices (not shown) may include amicrophone, joystick, game pad, satellite dish, scanner, or the like.These and other input devices are often connected to the processing unit21 through a serial port interface 46 that is coupled to the system bus23, but may be connected by other interfaces, such as a sound card, aparallel port, a game port or a universal serial bus (USB). A monitor 47or other type of display device is also connected to the system bus 23via an interface, such as a video adapter 48. In addition to the monitor47, personal computers may typically include other peripheral outputdevices such as speaker 45 and printers (not shown).

The personal computer 20 may operate in a networked environment usinglogic connections to one or more remote computers, such as a remotecomputer 49. The remote computer 49 may be another personal computer, aserver, a router, a network PC, a peer device or other network node, andtypically includes many or all of the elements described above relativeto the personal computer 20, although only a memory storage device 50has been illustrated in FIG. 1. The logic connections depicted in FIG. 1include a local are network (LAN) 51 and a wide area network (WAN) 52.Such networking environments are commonplace in offices, enterprise-widecomputer network intranets and the Internet.

When used in a LAN networking environment, the personal computer 20 isconnected to the local area network 51 through a network interface oradapter 53. When used in a WAN networking environment, the personalcomputer 20 typically includes a modem 54 or other means forestablishing communications over the wide area network 52, such as theInternet. The modem 54, which may be internal or external, is connectedto the system bus 23 via the serial port interface 46. In a networkenvironment, program modules depicted relative to the personal computer20, or portions thereof, may be stored in the remote memory storagedevices. It will be appreciated that the network connections shown areexemplary and other means of establishing a communications link betweenthe computers may be used.

For a better understanding of the present invention, a brief discussionof mouse message processing is now provided. FIG. 2A is a functionalblock diagram of computer 20 used with input device 42 in accordancewith one embodiment of the present invention. Mouse 42 illustrativelyhas right and left buttons and a depressible, rotatable wheel 103 therebetween. The block diagram of computer 20 shown in FIG. 2A includes anumber of the items discussed with respect to FIG. 1, and those itemsare similarly numbered. However, the block diagram in FIG. 2A also showsa number of components in greater detail which are used in processing amouse message. Computer 20 includes mouse driver 60, message hookprocedure 62, and focus application 64. To better understand theoperation of input device 42 in computer system 20 shown in FIG. 2A, thecomponents of that system are discussed in connection with a datastructure illustrated in FIG. 2B.

FIG. 2B illustrates a four-byte mouse packet 66 in a row and columnformat with bytes 68, 70, 72, and 74 shown in rows and the individualbits of each byte shown in columns. Byte 68 is the first byte providedby input device 42, byte 70 is the second byte, byte 72 is the thirdbyte, and byte 74 is the fourth byte. The columns of bits are organizedwith the least significant bits on the far right and the mostsignificant bits on the far left. Thus, column 76 includes the leastsignificant bits of each of the four bytes and column 78 includes themost significant bits of the four bytes.

Within mouse packet 66, first byte 68 includes left button bit 80, rightbutton bit 82, and middle button bit 84. A one in the left button bit 80indicates that the left button is depressed and a zero in left buttonbit 80 indicates the left button is not depressed. Similarly, a one inthe right button bit 82 or middle button bit 84 indicates that the rightbutton or the middle button, respectively, are depressed and a zero ineither of these bits indicates that their respective button is notdepressed.

Fourth bit 86 is set to a one.

Fifth bit 88 of byte 68 is the ninth bit of a 9-bit signed value that iscompleted by byte 70. The 9-bit value produced by the combination of bit88 and byte 70 represents the direction and magnitude of movement of themouse along the X-coordinate. Since the 9-bit value is in two'scomplement format, bit 88 indicates the direction of mouse movement suchthat if it has a value if zero, mouse movement is in a positive Xdirection and if it has a value of one, mouse movement is in thenegative X direction.

Sixth bit 90 of first byte 68 is the ninth bit of a 9-bit signed valuethat is completed by byte 72. The combination of bit 90 and third byte72 produces a value that indicates the magnitude and direction andmovement of the mouse along the Y coordinate. Since this value is atwo's complement signed value, bit 90 indicates the direction ofmovement along the Y coordinate such that if it has a value of one, themouse movement is in a negative Y direction and if it has a value ofzero, the mouse movement is in a positive Y direction.

Seventh bit 92 and eighth bit 94 of first byte 68 indicate whether the9-bit values formed by bit 88 and byte 70 and by bit 90 and byte 72,respectively, have incurred an overflow condition. This occurs when morethan nine bits of movement have been detected by the mouse. In thiscondition, the respective 9-bit value should be set to its maximummagnitude for the direction of movement.

The least significant four bits 96, 98, 100 and 101 of fourth byte 74represent the direction and magnitude of movement of wheel 103(illustrated in FIG. 2A). The value represented by bits 96-101 is asigned value wherein a positive value indicates wheel motion toward theuser and a negative value indicates wheel motion away from the user.

Bits 105 and 107 are the fifth and sixth bits of byte 74, respectively,and indicate closure of switches corresponding to the left and rightbuttons, respectively, of mouse 42. Thus, when bit 105 has a value ofone, the switch associated with the left button is closed indicatingthat the corresponding mouse button has been depressed. Bit 107 reflectsclosure of the switch associated with right mouse button in a similarfashion.

Bits 109 and 111 of fourth byte 74 are reserved for later use and areset to zero. Those skilled in the art will recognize that mouse packet66 illustrated in FIG. 2B and the serial interface 46 described beloware used in PS/2 and serial mouse connections. For universal serial bus(USB) connections, the mouse information is sent to the mouse driverusing publicly available USB protocols for mice.

In order to describe the processing of a conventional mouse message,reference is made to both FIGS. 2A and 2B. To initiate a mouse message,the user first manipulates mouse 42. Based on this manipulation, mouse42 generates a mouse packet that is passed to serial interface 46 andwhich is indicative of the manipulation event. When serial interface 46receives mouse packet 66, it converts the serial information in mousepacket 66 into a set of parallel packets and provides the parallelpackets to mouse driver 60. Mouse driver 60 creates a mouse messagebased on the manipulation event. The creation of the mouse message isidentical to the manner in which existing mice create mouse messages.

The mouse message is then transmitted to operating system 35. In oneillustrative embodiment, operating system 35 is a “WINDOWS NT®”, a“WINDOWS 95”, or a “WINDOWS 98®”, brand operating system (provided byMicrosoft Corporation of Redmond, Washington). Operating system 35includes a mouse message hook list that identifies a series of mousemessage hook procedures 62. When operating system 35 receives the mousemessage from mouse driver 60, it examines the mouse message hook list todetermine if any mouse message hook procedures have registeredthemselves with operating system 35. If at least one mouse message hookprocedure has registered itself with operating system 35, operatingsystem 35 passes the mouse message to the registered mouse message hookprocedure 62 that appears first on the list.

The called mouse message hook executes and returns a value to operatingsystem 35 that instructs the operating system to pass the mouse messageto the next registered mouse message hook.

The mouse message may, for example, represent a command to anapplication which owns the window currently under focus in computer 20.In that instance, the message hook procedure 62 issues the command tothe focus window application. In response, the focus window application64 performs the desired function.

After the message hook procedure 62 issues the command to the focusapplication 64, the message hook procedure 62 consumes the mouse messageby removing the message from the message chain. This is accomplished byreturning a value to operating system 35 which indicates to theoperating system that it should not pass the mouse message to any othermessage hook procedures.

FIG. 3 is a more detailed diagram, in partial block form and partialschematic form, illustrating an operator input device, such as mouse 42,in accordance with one embodiment of the present invention. Mouse 42includes housing 102, electromagnetic radiation source (which may simplybe a light source such as an LED) 104, aperture 106 defined in thebottom of housing 102, lens 108, image or pattern detector 110,controller 112, and current driver 114. In FIG. 3, mouse 42 is shownsupported relative to work surface 116. Pattern detector 110 can be anysuitable detector which is capable of detecting images or patterns frominformation carried by electromagnetic radiation impinging thereon andproviding a signal indicative thereof, and may be an artificial retinapattern detector as described in greater detail below, for example.

Light source 104 can be any suitable source of electromagnetic radiationwhich can be used to provide radiation for impingement on a pattern orimage and which can then be detected by pattern detector 110. In oneillustrative embodiment, light source 104 includes LED 118 and lens 120.Radiation emitted from an LED 118 is transmitted through lens 120 suchthat it passes through aperture 106 in housing 102 and impinges uponwork surface 116 which can have a predetermined pattern or imagethereon. The light then reflects off of work surface 116 toward lens108. Lens 108 collects the radiation reflected from surface 116 anddirects it to image detector (e.g., artificial retina) 110. It should benoted that lens 108 can be eliminated with the addition of lenses oneither LED 118, image detector 110, or both. Similarly, the lens cansimply be eliminated if the radiation is detectable by the detector,such that the image or pattern can be detected, without a lens.

Image detector 110 generates an image signal indicative of an image orpattern on work surface 116. The image signal is provided to controller112 which, in one illustrative embodiment, computes position informationbased on the image signal. The position information indicates movementof mouse 42 relative to work surface 116, as will be described in moredetail below. Position information is provided by controller 112 in theform of an information packet, through an output such as a cable (notshown), to computer 20 illustrated in FIGS. 1 and 2A. Mouse 42 may alsoprovide the output from controller 112 through a wireless transmissionlink such as infrared ultrasonic, or radiofrequency links. In anillustrative embodiment, the position information provided by controller112 is provided according to a conventional format, such as through aserial interface, a universal serial bus (USB) interface, or in anyother suitable interface format.

Image detector 110, in one illustrative embodiment, is an artificialretina manufactured by Mitsubishi Electric Corporation and includes atwo-dimensional array of variable sensitivity photo detectors (VSPDs)which operates in a known manner. Briefly, the VSPDs are formed by aside-by-side pair of diodes integrated onto and separated by asemiinsulated GaAs layer (pn-np structure). In one embodiment, the arrayis a 32×32 element array, but could be larger or smaller as desired. Thephoto detector current depends, both in sign and magnitude, on appliedvoltage. Such VSPDs exhibit an analog memory affect which storesconductivity information when a voltage is applied in the presence of anoptical write pulse. This information is retrieved by injecting anoptical readout pulse.

Image processing in such devices is based on optical matrix-vectormultiplication. An input image is projected onto the device as a weightmatrix. All VSPDs have one electrode connected along rows, yielding asensitivity control vector. Thus, the VSPD sensitivity can be set toarbitrary values in each row within a certain range. In addition, theremaining VSPD electrode is connected along columns, yielding an outputcurrent vector defined by the matrix vector product of the weight matrixtimes the sensitivity control vector.

In an illustrative embodiment, image detector 110 is controlled toperform edge extraction operations. The sensitivities of two adjacentdetector rows are set to +1 and −1, respectively, whereas all othersensitivities are set at 0. In this embodiment, the output current isproportional to the difference in light intensities of the two activerows. By shifting the control voltage pattern in a cyclical manner (0,+1, −1, 0, 0, etc.), the horizontal edges of the input image are sensed.Thus, the system operates in a time sequential and semi-parallel mode.

In one illustrative embodiment, mouse 42 also includes current driver114 which is coupled to source 104. In that embodiment, controller 112intermittently senses the intensity of the radiation generated by source104 and adjusts the current provided to source 104 through currentdriver 114. In other words, if the sensed intensity is lower than adesired range, controller 112 provides a feedback signal to currentdriver 114 to boost the current provided to source 104 in order toincrease the intensity of the electromagnetic radiation emanating fromsource 104. If, on the other hand, the intensity of the radiation ishigher than a desired range, controller 112 provides the feedback signalto current driver 114 to reduce the current provided to source 104 tothereby reduce the intensity of the radiation emitted from source 104.This may be done, for example, to reduce the overall power consumptionof mouse 42.

FIGS. 4 and 5A-5B illustrate the operation of mouse 42 in accordancewith one aspect of the present invention. Controller 112 includes A/Dconverter 122, control component 124, image matching component 126 andimage table 128. FIG. 4 also illustrates image detector 110 showing the32×32 VSPD (pixel) array 123 onto which the image from surface 116 isdirected. The entire viewing area 123 of image detector 110 iscoincident with the 32×32 pixel array. However, defined within theentire viewing area 123, is sample area 125. Sample area 125 is smallerthan the viewing area and includes, in one illustrative embodiment,approximately a 10 pixel by 10 pixel area centered generally around acenter 127 of viewing area 123. In the embodiment shown in FIG. 4, theimage on surface 116 is simply a grid pattern.

In operation, controller 112 first activates source 104 such thatradiation is impinged on work surface 116. Mouse 42 can operate on asurface having a predefined pattern (such as the grid structure) orwithout a predefined pattern. For example, substantially every surfacehas some irregularities or surface roughness, if only at the microscopiclevel. A commercially available artificial retina is capable ofresolving images which are only a few microns in size, assuming thelight source is powerful enough. Thus, the radiation is reflected off ofsurface 116 back to impinge on viewing area 123, carrying with itinformation indicative of either a predetermined pattern on surface 116or an image or pattern formed by the surface roughness on surface 116.

In the embodiment in which controller 112 is not looking for apredetermined pattern on work surface 116, the analog signal indicativeof the pattern formed by the surface roughness of surface 116 isprovided to A/D converter 122. A/D converter 122 converts the signalinto a digital value which is provided to control component 124. Controlcomponent 124 executes one of any number of suitable algorithms, such asthe above-identified edge extraction algorithm, to identify a pattern orimage from surface 116 which is reflected on sample area 125. Thisinformation is stored by control component 124 in a memory associatedtherewith. Control component 124 then waits for a predetermined time outperiod which is illustratively determined based on a maximum expectedvelocity of the mouse over surface 116. In one illustrative embodiment,the time out duration is approximately 1-10 milliseconds.

After the time out period, control component 124 reactivates source 104(if it has been de-activated), and determines whether the image withinsample area 125 has moved. Movement of the image within sample area 125is indicative of relative movement between mouse 42 and surface 116.Based on detected movement, control component 124 provides positioninformation in a customary and acceptable format (such as packet 66illustrated in FIG. 2B) at an output (such as through a cable). Thisinformation is used for any number of things, including the movement ofa mouse cursor on the computer display.

After movement of the image or pattern within viewing area 125 isdetected, a new image or pattern within sample area 125 is detected.Image data, indicative of the new image, is then stored by controlcomponent 124. Control component 124 then again waits for another timeout period and determines whether the new image has moved. This processcontinues such that mouse 42 continues to provide position informationindicative of the relative movement of mouse 42 and work surface 116.

Detection of movement of the image in sample area 125 can beaccomplished in a number of fashions. Such techniques includecross-correlation, circular harmonics, pattern moment characteristics,and chord histogram detection. Rotational movement of mouse 42 withrespect to surface 116 can also be detected, if desired. In oneillustrative embodiment, lens 120 is shaped to transmit radiation in anelongate, or elliptical, pattern. Therefore, if the mouse is rotated,rotation of the elliptical shape can be detected as well. All of thesetechniques are described in greater detail in the above-identifiedco-pending U.S. patent application.

Controller 112 can also be configured to detect a predetermined pattern(such as the grid structure) within viewing area 123, and to detectmovement of the predetermined pattern relative to mouse 42. For example,FIGS. 5A and 5B illustrate the movement of a predetermined patternwithin sample area 125. In the embodiment illustrated in FIGS. 5A and5B, the predetermined grid pattern (shown with vertical lines only forthe sake of clarity) is formed of alternating blue and red lines suchthat the radiation reflected back to image detector 110 carries with itinformation of whether blue or red grid lines are within sample area125. Of course, the alternating grid lines could be formed ofalternating thickness, of different shades, or have other distinguishingcharacteristics which allow control component 124, when it detects theimage, to identify the alternating grid lines.

In FIG. 5A, two blue lines 132 and 134 are within sample area 125, andone red line 136 is within sample area 125, between blue lines 132 and134. After movement of the mouse, a new pattern resides within samplearea 125, as illustrated in FIG. 5B. The new pattern shows that lines132, 134 and 136 have shifted to the left relative to sample area 125,and that another red line 138 has entered the right side of sample area125.

The sample rate of image detector 110 is illustratively high enough,again based on the expected maximum velocity of the mouse, that mouse 42cannot be moved one full grid spacing before a new image is taken. Inone illustrative embodiment, control component 124 assumes apredetermined, constant distance between grid lines. In that way,control component 124 simply needs to track the number of grid lineswhich pass through sample area 125, and the direction of movement ofthose grid lines, in order to determine the distance and direction whichmouse 42 has moved relative to surface 116. Of course, this isillustratively carried out for grid lines in both the X and Ydirections. Control component 124 then generates a mouse packet, similarto that illustrated in FIG. 2B, which is indicative of the movement ofmouse 42 relative to surface 16.

VARIABLE RESOLUTION OPERATION

FIG. 6 illustrates a mouse pad, or surface, 116 having four differentzones 140, 142, 144 and 146 disposed thereon. Zone 140 on surface 116has no predetermined pattern thereon. Section 142 has a grid linepattern of uniformly spaced grid lines. Zone 144 has a high resolutiongrid line pattern in which the grid line are uniformly spaced, but arespaced closer than those in zone 142. Zone 146 also has a grid linepattern, but the pattern in non-orthogonal in that the spacing betweenthe grid lines varies from one side of zone 146 to another side of zone146. Zones 140-146 allow mouse 42 to operate in a variable resolutionmanner without the need for changing a device driver, and withoutinvoking a control panel applet.

When mouse 142 is over zone 140, control component 124 provides positioninformation based on random (or in any case non-predetermined) patternsas discussed above. In other words, control component 124 takes periodic“snapshots” of patterns or images identified within sample area 125based on surface roughness in zone 140, and determines movement of thoseimages relative to sample area 125 to provide position information.

However, when mouse 42 is moved over resolution zone 142, controlcomponent 124 identifies the grid line structure beneath mouse 42. Sincethe grid lines are illustratively far more pronounced than the surfaceroughness of surface 116, control component 124 can easily identify thatmouse 42 is over a zone which has a predetermined pattern thereon.Therefore, when mouse 42 is over zone 142, control component 124 simplyprocesses the images as discussed above with respect to FIGS. 5A and 5B,counting the grid lines along both axes which pass through sample area125, to determine movement of mouse 42.

Similarly, when mouse 42 is moved over high resolution zone 144, controlcomponent 124 can quickly determine that mouse 42 is over a zone havinga predetermined pattern thereon, and operates in the mode discussed withrespect to zone 142. Recall that control component 124 is configured toassume a predetermined distance between grid lines. While grid lines inzone 144 are uniformly spaced, they are spaced much closer than those inzone 142 (e.g., the grid lines in zone 144 have a spacing which is halfthat of the grid lines in zone 142). Control component 124 is simplycounting the number of lines which pass through sample area 125.Therefore, for a given amount of movement of mouse 42 relative to zone144, control component 124 will provide position information indicatingthat mouse 42 has moved twice as far as the same amount of relativemovement of mouse 42 with respect to zone 142. By simply moving mouse 42from zone 142 to zone 144, the user can effectively half the scale ofmouse 42 and double its resolution, without changing any software andwithout accessing the control panel.

Zone 146 also has a grid pattern disposed thereon. However, unlike zones142 and 144, the spacing between the grid lines in zone 146 variesacross the zone. As illustrated in FIG. 6, the spacing of the grid linesin a central region of zone 146 is the largest, while the spacingbetween the grid lines on either end of zone 146 (and top to bottom)decreases toward the edges of zone 146. As described with respect tozones 142 and 144, control component 124 detects that mouse 42 is over aregion having a predefined pattern thereon. The resolution and scale ofmouse 42 changes as it is moved across zone 146, based upon the changein the spacing of the grid lines in zone 146. Therefore, the user canconvert mouse 42 into a variable scale and variable resolution inputdevice simply by placing the mouse over zone 146, again without makingany changes to the device driver and without invoking a control panelapplet.

DETECTION OF SPECIALLY CODED IMAGES

Controller 112, in conjunction with image sensor 110, is illustrativelyconfigured to detect substantially any image on surface 116, whether itis a predefined image, or a random image indicative of surfaceroughness. Therefore, mouse controller 112 and image detector 110 canalso detect specially coded images, which have a special meaning, andwhich are disposed on surface 116.

FIG. 7 illustrates a very simple coded pattern which can be repeated ina given zone on surface 116.

In one illustrative embodiment, the coded patterns are identified bycontrol component 124 using a simple projection technique illustrated inFIG. 8. The array of pixels in image sensor 110 can be controlled toprovide an output which represents a sum of the active pixels in eachrow, and in each column. This information is projected on the XY axes asillustrated in FIG. 8.

In other words, the pattern illustrated in FIGS. 7 and 8 is an invertedtriangular pattern which, when reflected on the detector array,activates three pixels at its base and, one at its inverted pinnacleend. The pattern can be projected onto a single axis and represented bya voltage signal 150 on the X axis and a voltage signal 152 on the Yaxis. Signal 150 has a first level 154 which is indicative of no activepixels in the viewing area. Signal 150 also has a second level 156 whichis indicative of a single active pixel in the viewing area. Further,signal 150 has a third level 158 indicative of two active pixels in theviewing area. Thus, signal levels 154, 156 and 158 represent a sum ofthe signals generated by the active pixels, associated with the image,when projected on the X axis.

Signal 152 illustrates projection of the image onto the Y axis. Signal152 has a first level 160 which is indicative of three active pixels andsecond and third levels 162 and 164 which are indicative of two activepixels, and one active pixel, respectively.

Upon receiving these digitized signals, control component 124 identifiesthe coded pattern and provides it to matching component 166. Matchingcomponent 166 accesses an image table 128 which stores data indicativeof all coded images which can be recognized by controller 112. Matchingcomponent 126 uses any suitable, and preferably simple, matchingalgorithm to match the image identified by control component 124 to apredetermined coded image stored in image table 128. Illustratively,image table 128 not only stores data indicative of the images to berecognized but also includes a value associated with the recognizedimage, and which can be used in the mouse packet generated by controlcomponent 124 in passing the desired information back to computer 20.

In order to generate the mouse packet, control component 124 preferablygenerates a new mouse packet generally illustrated by the number 170 inFIG. 9. Packet 170 is similar to packet 66 illustrated in FIG. 2B,except that it includes an additional byte 172 of information. Byte 172,as with the other bytes in packet 170, preferably includes eight bits ofinformation which are used to encode the fact that control component 124has, indeed, recognized a predetermined image which resides in imagetable 128, and which also includes the value in image table 128associated with the coded pattern which has been identified.

By way of example, the coded pattern illustrated in FIG. 7 can berepetitively disposed on surface 116 to indicate that computer 20 is tooperate in a certain mode, or is to change modes. In one illustrativeembodiment, when the coded pattern illustrated in FIG. 7 is detected, afunction/mode change value associated with that pattern in image table128 indicates that computer 20 is to operate in a vision impaired mode,in which all fonts are dramatically increased in size, above a nominallevel.

FIGS. 10A and 10B depict a flow diagram illustrating the operation ofmouse 42 and computer 20 under such circumstances. First, the usermanipulates the mouse as indicated by block 174. In other words, theuser simply moves or places the mouse over a region or zone of surface116 which contains the repetitive coded patterns. Next, image sensor 110detects the image and passes the data indicative of the detected imageto control component 124. Control component 124 then identifies theimage, as discussed above, and passes the image to matching component126. This is indicated by blocks 176 and 178.

Matching component 126 accesses image table 128 and matches theidentified image (if possible) to an image contained in image table 128.This is indicated by block 128. Matching table 126 then passes thefunction/mode change value associated with the matched image from imagetable 128 to control component 124. Control component 124, in turn,generates the mouse packet with the function/mode change value containedin byte 5 of the packet. This is indicated by block 182.

Control component 124 then passes the packet to serial interface 46, asindicated by block 184. Serial interface 46 converts the serial mousepacket into parallel mouse packet information and provides thatinformation to mouse driver 60. Mouse driver 60 examines the informationin byte 5 and creates a mouse message based on the function/mode changeinformation which is in turn based on the predetermined coded patternwhich has been identified. This is indicated by blocks 186 and 188.Mouse driver 60 passes the mouse message to operating system 35, asindicated by block 190.

It should be noted that the mouse message can be a specialized messageintended for the operating system. For example, in order to operate in avision impaired mode, the mouse message may indicate to the operatingsystem that the font size needs to be increased. Determination ofwhether mouse message is a specialized message is indicated by block192. If the mouse message is a message intended for the operatingsystem, the operating system takes the necessary steps to change theoperating mode of computer 20, based upon the information in the mousemessage. This is indicated by block 194. In the instance where computer20 is to change to a vision impaired mode, the operating system invokesa control panel change of the font size to a desired level, such as bycalling an API. The operating system then consumes the mouse message.This is indicated by blocks 194 and 196.

If, at block 192, it is determined that the mouse message is not aspecialized message intended for the operating system, the operatingsystem simply passes the mouse message to registered mouse messagehooks. This is indicated by block 198. The message hook may beregistered by an application which is configured to utilize theinformation contained in the mouse message. Such information may, forexample, indicate that a command is to be passed to the applicationwhich owns the window currently under focus in computer 20. Therefore,the message hook executes by identifying the focus, as indicated byblocks 200 and 202. The message hook then issues the desired command tothe focus application and consumes the mouse message. This is indicatedby blocks 204 and 196.

While the above discussion has proceeded with respect to changing theoperating mode of computer 20 from a normal vision mode to a visionimpaired mode, the present technique can be used to make substantiallyany changes to the operating mode of computer 20. Change of font size isbut one example. It should also be noted that, while mouse 42 canidentify coded images on surface 116, it can simultaneously provideposition information based either on movement of the coded images withinsample area 123, or based on movement of images created by the surfaceroughness of surface 116 within sample area 123, both of which aredescribed above. In that case, mouse packet 170 not only includesinformation contained in byte 5, but also includes position informationindicative of the movement of mouse 42 in the X and Y directions. Ofcourse, mouse 42 can also simultaneously provide information in packet170 which is indicative of the actuation of buttons and the rotation ofthe wheel on mouse 42.

Further, mouse pads are illustratively used which contain differentcoded messages on opposite sides thereof. In other words, the impairedvision code can be provided on one side of the mouse pad, and repeatedacross the entire surface of the mouse pad, along with grid lines havingvarious zones such as those shown in FIG. 6. In that instance, a singleside of the mouse pad provides an indication to computer 20 that it isto operate in the vision impaired mode, and also provides the user withthe variable resolution capability discussed above with respect to FIG.6. At the same time, the opposite side of the mouse pad can contain thesame variable resolution zones (or any other suitable pattern) withoutthe vision impaired coded symbols thereon. Thus, if the user is visionimpaired, the user can simply flip over the mouse pad, place mouse 42 onthe pad, and computer 20 will automatically switch to a vision impairedmode.

The present invention can also be used to accomplish other desiredfunctions. For example, mouse 42 can also be used as user input devicefor inputting a user ID or password to perform a logon operation. FIG.11 is a block diagram similar to that shown in FIG. 4, and similar itemsare correspondingly numbered. However, FIG. 11 illustrates a differentimage 210 which is disposed on surface 116. In the embodimentillustrated in FIG. 11, image 210 is illustrated as a barcode. However,any other recognizable image can be used. Image 210, in one embodiment,is placed on a personalized mouse pad, or an ID badge which a usercarries for security purposes, or on another similar personalized item.

In some conventional computer systems, a logon procedure is requiredbefore the user is permitted access to certain aspects of the computersystem. When a computer is booted up, the boot-up sequence may requirethe operating system to call a logon application which generates a userinterface requiring the user to input identifying information, such as aname or ID number, as well as a password. Based on this user inputinformation, the logon application can allow the user full access of thecomputer system, only partial access (where different levels of securityare implemented), or no access at all (such as where the user inputinformation reflects that the user is not authorized to have any accessto the system).

For example, some financial or billing system applications only allowauthorized users to have access to certain data bases containingconfidential financial information. Similarly, in a network environment,some logon applications will control access to the network drives basedon the user's identity and security level. Further, automatic logonprocedures implemented in systems utilizing the “WINDOWS” brandoperating systems require the user to reboot the computer (such as byexecuting the CTL-ALT-DEL key sequence) before the logon procedure willbe executed. Such logon operations can be somewhat cumbersome.

Therefore, the present invention provides a system by which logon can beaccomplished simply by placing mouse 42 over a coded image whichcontains the user's personal logon information, or by scanning mouse 42across an image (such as the barcode illustrated in image 210) whichencodes the user's personal logon information.

Where the coded image is small enough that it can be recognized bycontroller 112 without moving mouse 42 (i.e., where it is small enoughthat it will appear within viewing area 123 in its entirety, the imageis simply processed as any other coded image, as described above. Inother words, the image is captured by image detector 110, provided toA/D converter 122 which converts the image signal into a digital signaland is passed to control component 124. Control component 124 thenidentifies the image and provides it to matching component 126 whichmatches the identified image to a corresponding image stored in imagetable 128. In that instance, image table 128 also includes an associatedimage value which identifies the image as a user's logon information.

This information is provided back to control component 124 whichgenerates the mouse packet and provides the mouse packet, through serialinterface 46, to mouse driver 60. Mouse driver 60, in turn, generates amouse message which is transmitted to operating system 35. In anembodiment in which operating system 35 handles logon operations, themouse message is designated as a specialized mouse message for operatingsystem 35. In another embodiment in which a separate application handlesthe logon procedures, the mouse message is simply designated as a normalmouse message which is to be passed to registered message hookprocedures 62. In that case, operating system 35 passes the mousemessage to the message hook procedures until it reaches the logonapplication. The logon application issues commands to other componentsof computer 20 to configure computer 20 to allow a desired level ofaccess to the user, or to deny the user access and simply issue an errormessage to the user.

In an embodiment in which mouse 42 is scanned across image 210 (such aswhere image 210 is a barcode) operation is generally indicated by theflow diagrams illustrated in FIGS. 12A and 12B. It is worth noting atthe outset that mouse 42 can be configured to receive a select inputfrom the user (such as depression of one of the buttons) indicating thatthe user wishes to enter a logon mode.

The user then scans mouse 42 across image 210 as indicated by block 212.By scanning mouse 42 across image 210, a sequence of discrete imageswill be generated which are indicative of the information encoded inbarcode image 210. Image detector 110 captures this image sequence andprovides it, in turn, to A/D converter 122 which provides digitalinformation representative of the image sequence to control component124. This is indicated by block 214.

Control component 124 identifies each image in the sequence of images,as indicated by block 216 and provides the sequence of images to amatching component 126. Matching component 126 matches the imagesequence to an image sequence stored in image table 128, as indicated byblock 218. Control component 124 receives the associated informationfrom image table 128 and generates the mouse packet including logoninformation associated with the matched image sequence. This isindicated by block 220. Control component 124 passes the mouse packet toserial interface 46 as indicated by block 222. Serial interface 46 thenconverts the serial mouse packet into parallel mouse packet informationand provides the parallel mouse packet information to mouse driver 60,as indicated by block 224.

Driver 60 creates a mouse message based on the logon event informationreceived, as indicated by block 226, and passes the mouse message tooperating system 35 as indicated by block 228. In the embodiment inwhich the logon procedures are to be handled by an application separatefrom operating system 35, the logon application must have alreadyregistered itself with the operating system 35 as a mouse message hook.This is indicated by block 230.

Operating system 35 then passes the mouse message to a next registeredmouse message hook as indicated by block 232 and the message hookprocedures are executed as indicated by block 234. In one illustrativeembodiment, the message hook simply issues commands to the logonapplication requesting that the logon application verify theidentification and password information in the mouse message. This isindicated by block 236. Commands are then issued from the logonapplication which configure computer 20 to allow appropriate access tothe user. This is indicated by block 238. The message hook associatedwith the logon application then consumes the mouse message as indicatedby block 240. It should be noted that, while the above discussion hasproceeded with respect to a logon operation, those skilled in the artwill recognize that a technique implemented according to the presentinvention can be used to perform any other desired functions, or toissue commands to any other applications, as well.

GENERATION OF CUSTOM TEMPLATES

The present invention can also be used to generate custom templates, orprinted surfaces, 116 which include customized coded images thereon(printed on paper, Mylar, etc.) which act to customize the system for aspecific user or a specific need. For example, some game applicationsexpose additional functionality as the user reaches higher levels in thegame. In other words, as the user reaches a certain score, or traversesa certain screen, the game application provides additional “lives” or“ammunition” or provides the user with access to additional “weapons” orother tools or instruments which can be used in the game. Similarly, insome such applications, after the user reaches a certain level, theapplication triggers a different response through the utilization of anexisting tool or implement. The present invention can be used to enhancethe functionality of such games.

In order to enhance functionality, the application is configured toprint out a customized template for placement on surface 116. Thetemplate includes customized encoded images which are read by mouse 42and result in the creation of a mouse message which is interpreted bythe application to modify the functionality of the application. Thisprocess is generally indicated by the flow diagram set out in FIG. 13.

First, the application receives a trigger event to print a newcustomized template containing the customized encoded images. This isindicated by block 242. As discussed above, this event can simply bethat the user reaches a predetermined level in a game. The trigger eventcan also correspond to the user entering or scanning in a secret codewhich has been revealed by the application to the user. In theembodiment in which the user scans such a secret code with mouse 42, thecode is transmitted to the application using (for example) the protocolidentified above with respect to FIGS. 10A and 10B.

The application then issues commands to print out the applicationspecific, customized template. This is indicated by block 244. Also, ofcourse, the application must register any other required mouse messagehooks with operating system 35, such that it will receive mouse messagescontaining information indicative of the customized coded images on thenew template. This is indicated by block 246. The new template is thenplaced on surface 116 and used by the user.

It should be noted that while the above discussion has proceeded withrespect to a games application, the invention is not so limited. Thoseskilled in the art will recognize that the invention can be utilized toprint out any template which is customized to an application, regardlessof the application in which it is used.

USE OF A MOUSE 42 AS AN ABSOLUTE POSITION DEVICE

The present invention can also be utilized to selectively convert mouse42 from a relative position device to an absolute position device. Thisis illustrated in FIG. 14. FIG. 14 illustrates mouse pad or template 248which has three separate zones 250, 252, and 254 disposed thereon. Inone embodiment, zone 250 either has no predetermined pattern thereon, orhas a grid structure pattern, or other similar pattern such as thatdescribed with respect to FIGS. 5A and 5B. Thus, when mouse 42 is overzone 250, it simply acts as a relative positioning device.

Zone 252 has a repetitive code disposed thereon. In other words, zone252 has a number of cells 256, each cell containing the same codedsymbol 258. The repetitive coded area in zone 254 can be used, asdiscussed above, to place computer 20 in a desired mode of operation,and can also be used to generate relative position informationcorresponding to movement of mouse 42 over zone 252.

However, template 248 also includes zone 254 which contains a pluralityof cells 260, each encoded with a unique coded image 262. Since eachcoded image 262 in cells 260 is different from the other encoded images262 in other cells 260, placement of mouse 42 over zone 254 can be usedto convert mouse 42 into an absolute positioning device.

Each of the unique coded images 262 are stored in image table 128. Eachof the coded images also has a value associated therewith (in imagetable 128) which is indicative of an absolute position of mouse 42within zone 254. Therefore, when a coded image 262 is identified bycontrol component 124, it is passed to matching component 126 whichmatches the coded image to a corresponding image in image table 128.Image table 128 then provides the absolute position information tocontrol component 124 which generates the mouse packet passed tocomputer 20, including the absolute position information. This absoluteposition information can be used to place a cursor on the display screenof computer 20 at a predefined position which corresponds to theabsolute position information, regardless of the relative movement ofmouse 42. In that way, mouse 42 can be picked up, off of template 248,and set down at another location within zone 254. The cursor will thenbe moved to another position on the display screen corresponding to theabsolute position read from zone 254 over which the mouse is thenplaced.

GENERATION OF IMAGES IN IMAGE TABLE 128

It will be appreciated that, prior to matching identified images toimages which reside in image table 128, the images must be somehowgenerated and placed in image table 128. FIGS. 15A, 15B and 16illustrate a number of embodiments for accomplishing this.

At the outset, the images can be preformed images which are simplyloaded into computer 20. The preformed images are then downloaded tocontrol component 124 in controller 112 which places the images in imagetable 128. Downloading the images can be accomplished using any suitableprotocol. For example, an image loading component in computer 20 canaccess the preformed images stored in computer 20 and generate imagepackets which are passed to operating system 35. In that instance, mousedriver 60 is provided with an image packet transmission component whichreceives the image packets from operating system 35 and provides them inparallel, to serial interface 46. Serial interface 46 then serializesthe packets and provides them, through a suitable link, to controlcomponent 124 which simply places the images in image table 128. Theseimages are then used by matching component 126 in matching imagescaptured by image detector 110 and identified by control component 124.

While the preformed images can take substantially any form, in oneillustrative embodiment, they are orientation compensating codes. Forinstance, the codes will be recognizable by control component 124,regardless of the angular orientation of mouse 42, in the plane ofsurface 116 over which it is disposed. Therefore, if the user slightlyrotates mouse 42 in the plane of surface 116 as the user moves mouse 42relative to surface 116, the coded image will be constructed such thatit can be identified regardless of the particular orientation of mouse42.

FIG. 15B is one illustrative embodiment of such a coded image. FIG. 15Bshows a coded image 264 which contains a pair of concentric circles 266and 268, and an orientation marker 270. Image 264 also contains aplurality of coded image cells 272 which contain information indicatedby a coded message 264. When the image signals indicative of image 264are provided to control component 124, control component 124 uses asimple algorithm to identify concentric circles 266 and 268, and marker270. Based on the location of marker 270, control component 124 caneasily determine the orientation of image 264 relative to mouse 42, andcan then proceed to examine the remainder of image 264 for theinformation in cells 272.

While a wide variety of such orientation compensating codes can be used,and while orientation compensating codes do not necessarily need to beused in all embodiments of the present invention, some such orientationcompensating codes are commercially available and are designated by theterm USS-MaxiCode system. Codes utilizing this system also exhibitcertain fault tolerance characteristics which may be desirable.

Not only can the coded images be preformed and downloaded from computer20 to mouse 42, but they can be learned by computer system 20 and mouse42 as well.

FIG. 15A is a functional block diagram illustrating computer 20. FIG.15A is similar to FIG. 2A, and similar items are correspondinglynumbered. However, FIG. 15A illustrates that computer 20 is alsoprovided with a learn mode component 274. Learn mode component 274 isillustratively associated with a learn mode application which hasregistered a message hook with operating system 35.

FIG. 16 is a flow diagram illustrating operation of mouse 42 andcomputer 20 in the learn mode. First, the user initiates the learn mode.This can be done simply by placing the mouse over a precoded pattern onthe mouse pad, by depressing an actuator button on the mouse, bydepressing a key or key sequence on the keyboard, etc. In response tolearn mode initiation, operating system 35 calls learn mode component274 which provides a user interface to the user indicating theparticular function or mode change which is to be assigned to the nextcoded image which is captured by mouse 42 and learned. Initiation of thelearn mode in specifying the function/mode change is indicated by blocks276 and 278 in FIG. 16.

The user then scans the mouse over a selected area on surface 16 whichcontains the coded image to be captured. Alternatively, the user cansimply set the mouse 42 on that area, if the coded images are repeatedon surface 116 and are spaced closely enough together that, no matterwhere the user sets mouse 42, at least one of the coded images will becaptured by image detector 110. This is indicated by block 280.

The images are then captured by image detector 110 and image dataindicative of the image is provided by control component 124 in mouse42. A mouse packet is transmitted to serial interface 46 and to mousedriver 60 where a mouse message carrying the image data is generated andprovided to operating system 35. This is indicated by blocks 282 and284. Operating system 35 transmits the mouse message to message hookprocedures which have registered with operating system 35, such that themouse message will eventually be provided to learn mode component 274.This is indicated by block 286.

Learn mode component 274 characterizes the image by assigning a patternkey or pattern signature key to the image which can be used by controlcomponent 124 in identifying the image, and by matching component 126 inmatching the image to other images stored in image table 128. This isindicated by blocks 286 and 288. Learn mode component 274 thenassociates the pattern key generated with a value indicative of thefunction or mode change which is to be represented by the coded image.This value is associated with the image key to form an image table entryfor entry in image table 128 in mouse 42. This is indicated by block290.

The image table entry is then transmitted back to mouse 42 using anysuitable protocol, such as that set out for sending preformed codedimages to mouse 42 from computer 20. This is indicated by block 292. Theimage table entries which are sent to mouse 42 are also sent to a memorylocation on computer 20 such that they can later be downloaded toanother mouse 42, for example, when one mouse is switched for anothermouse during the operation of computer 20.

It should be noted that, while the present discussion of the learn modehas proceeded with respect to only capturing a single image to beassociated with a given function, other methods could be used as well.For example, the learn mode component can instruct the user to againplace the mouse over the same coded image to be learned such that twoinstances of the same coded image can be captured. The two instances arethen averaged by learn mode component 274 in order to obtain an averagevalue indicative of the coded message to be learned. This process isillustratively repeated a number of times in order to help reduce theaffects of noise, or other anomalies, on the image capturing process.

CONCLUSION

It can thus be seen that the present invention provides a mechanism bywhich pattern information can be provided to a computer. The patterninformation can be used by the computer for a wide variety of purposes.The computer can be configured such that operational characteristics ofthe computer can be changed based on the pattern information. Theoperational characteristics can represent substantially any change inhow the computer operates. The change in operational characteristics canbe referred to as a change event generated based on images, (whichinclude partial images or patterns) read from surface 116. The changeevent can reflect, for example, changes in behavioral characteristics ofthe computer input device, operational modes of the computer to which itis attached, operational characteristics of the operating system of thecomputer, commands to applications, or the input of user identificationinformation. Since the change event is generated and provided tocomputer 20 based on what is “read” from a template or pattern providedon surface 116 by an optical tracking device, such as a mouse, thepresent invention overcomes significant disadvantages associated withprior systems in which software drivers or control panel applets wererequired to be manipulated or invoked in order to make such changes.

Although the present invention has been described with reference topreferred embodiments, workers skilled in the art will recognize thatchanges may be made in form and detail without departing from the spiritand scope of the invention.

What is claimed is:
 1. A computer input system for providing inputinformation to a computer, comprising: a computer input deviceconfigured to provide position information based on a position of thecomputer input device relative to a surface, the computer input devicecomprising a pattern detector configured to detect a pattern on thesurface and a controller coupled to the pattern detector; and a patterndisposed on the surface, the pattern having at least first and secondpredetermined pattern portions thereon, the controller generating achange event based on the pattern detector switching between detectingthe first predetermined pattern portion and detecting the secondpredetermined pattern portion; wherein the controller is configured toprovide relative position information indicative of relative movement ofthe computer input device relative to the pattern when the patterndetector is detecting the first predetermined pattern portion andabsolute position information indicative of an absolute position of thecomputer input device with respect to the pattern when the patterndetector is detecting the second predetermined pattern portion.
 2. Thecomputer input system of claim 1 wherein the controller is configured tochange the position information provided for a given distance ofrelative movement of the computer input device with respect to thepattern, based on whether the pattern detector is then detecting thefirst predetermined pattern portion or a third predetermined patternportion.
 3. The computer input system of claim 2 wherein the firstpredetermined pattern portion comprises a first plurality ofpredetermined images spaced a first distance apart and wherein the thirdpredetermined pattern portion comprises a second plurality ofpredetermined patterns spaced a second distance apart.
 4. The computerinput system of claim 3 wherein the first plurality of predeterminedimages comprises a first plurality of grid lines spaced the firstdistance apart and wherein the second plurality of predetermined imagescomprises a second plurality of grid lines spaced the second distanceapart.
 5. The computer input system of claim 2 wherein the firstpredetermined pattern portion comprises a first plurality ofpredetermined images spaced from one another by a distance which variesacross the pattern.
 6. The computer input system of claim 1 wherein thesecond predetermined pattern portion comprises a plurality of uniquepredetermined images spaced from one another on the surface.
 7. Acomputer input device providing input information to a computer,comprising: a position transducer configured to provide positioninformation based on movement of the computer input device relative to asurface, the position information for use in positioning a displayelement on computer screen; a pattern detector configured to detect apredetermined image of an optical pattern on the surface; and acontroller coupled to the pattern detector and the position transducer,the controller being configured to generate a message to the computerindicative of the position information to position the display elementand indicative of an operating mode change based on the predeterminedimage of an optical pattern detected.
 8. The computer input system ofclaim 7 wherein the controller provides mode change information based onthe pattern detector switching between detecting a first predeterminedpattern portion and a second predetermined pattern portion.
 9. Thecomputer input system of 8 wherein the first predetermined patternportion comprises at least one predetermined image indicative of a firstcomputer operating mode and wherein the second predetermined patternportion comprises a second predetermined image indicative of a secondcomputer operating mode.
 10. The computer input system of claim 7wherein the controller is configured to generate mode change informationindicative of command information for issuing a command to anapplication running on the computer, the controller generating thecommand information based on the pattern detector detecting at least oneof the first and second predetermined pattern portions.
 11. The computerinput system of claim 10 wherein the application comprises a logonapplication and wherein the command information is indicative of a logoncommand and user authentication information, the logon applicationcontrolling access provided to a user based on the logon command and theuser authentication information.
 12. The computer input system of claim11 wherein the surface comprises a portion of a user identificationbadge.
 13. The computer input system of claim 1 wherein the first andsecond predetermined pattern portions are learned patterns which arelearned by the controller in response to initiation of a learn mode ofoperation.
 14. The computer input system of claim 1 wherein thecontroller is housed within the computer.
 15. A method of providing auser input to a computer, the method comprising: placing a user inputdevice over a first portion of a surface having a first predeterminedpattern disposed thereon; detecting a first image indicative of thefirst predetermined pattern; providing first pattern information to thecomputer indicative of detection of the first image; placing the userinput device over a second portion of the surface having a secondpredetermined pattern disposed thereon; detecting a second imageindicative of the second predetermined pattern; providing second patterninformation to the computer indicative of detection of the second image;and providing position information to the computer indicative ofmovement of the user input device relative to the surface, the positioninformation being used to position a display element on a computerscreen.
 16. The method of claim 15 and further comprising: controllingthe computer based on the first and second pattern information.
 17. Themethod of claim 16 wherein controlling comprises: changing one of anoperating parameter of the user input device, and an operational mode ofthe computer, based on a change from detecting the first image todetecting the second image.
 18. The method of claim 16 wherein thecontrolling step comprises: changing a mode of operation of the computerbased on a change from detecting the first image to detecting the secondimage.
 19. The method of claim 18 wherein the step of detecting a firstimage comprises: capturing the first image with an image detector; andmatching information indicative of the first image to informationindicative of a first stored image stored in an image store.
 20. Themethod of claim 19 wherein the step of changing a mode of operation,comprises: providing a mode value associated with the first storedimage, the mode value being indicative of a first mode of operation ofthe computer.
 21. The method of claim 18 wherein the step of detecting asecond image comprises: capturing the second image with an imagedetector; and matching information indicative of the second image toinformation indicative of a second stored image stored in an imagestore.
 22. The method of claim 21 wherein the step of changing a mode ofoperation, comprises: providing a mode value associated with the secondstored image, the mode value being indicative of a second mode ofoperation of the computer.
 23. The method of claim 16 wherein thecontrolling step comprises: changing an operating parameter of the userinput device.
 24. The method of claim 23 wherein the user inputcomprises position information, and wherein the controlling stepcomprises: changing the position information from relative positioninformation indicative of movement of the user input device relative tothe surface to absolute position information indicative of an absoluteposition of the user input device with respect to the surface, based onwhether the user input device is detecting the first image or the secondimage.
 25. A method of controlling a computer, comprising: providing acomputer input device moveable relative to a surface; detecting apredetermined image of an optical pattern on the surface with thecomputer input device; and configuring the computer to operate accordingto an operational mode based on the optical pattern detected on thesurface, and providing position information to the computer based onmovement of the computer input device relative to the surface toposition a display element on a computer screen.
 26. The method of claim25 wherein the step of providing position information comprises:detecting a substantially random image on the surface; moving the userinput device relative to the surface; and detecting a change in relativeposition of the substantially random image relative to the user inputdevice.
 27. A method of controlling a computer based on inputinformation provided by a user input device, comprising; placing theuser input device over a surface; generating position information basedon movement of the user input device relative to the surface;positioning a display item on a computer screen based on the positioninformation generated from the input device; detecting a predefinedimage of an optical pattern on the surface; and issuing a command to anapplication program running on the computer based on the predefinedoptical pattern detected on the surface.
 28. A method of controlling acomputer based on input information provided by a user input deviceuseable for generating position information based on movement of theuser input device relative to a surface and positioning a display itemon a computer screen based on the position information generated fromthe input device, the method comprising: detecting a predefined image ofan optical pattern on a surface; and controlling a procedure to log onto the computer based on the predefined image of the optical patterndetected.
 29. A computer input device for providing input information toa computer, comprising: a position transducer configured to provideposition information based on a position of the computer input devicerelative to a surface, the position information being used to position adisplay element on a computer screen; a pattern detector configured todetect first and second predefined images of optical patterns on thesurface; and a controller coupled to the pattern detector and beingconfigured to generate pattern information for transmission to thecomputer to control an operating mode of the computer, the patterninformation being indicative of whether the pattern detector isdetecting the first predefined image of an optical pattern or the secondpredefined image of an optical pattern.
 30. The computer input device ofclaim 29 wherein the controller is configured to generate a firstinformation packet including a pattern portion containing first patterninformation indicative of the pattern detector detecting the firstpredefined image of an optical pattern.
 31. The computer input device ofclaim 29 wherein the controller is configured to generate a secondinformation packet including a pattern portion containing second patterninformation indicative of the pattern detector detecting the secondpredefined image of an optical pattern.
 32. The computer input device ofclaim 31 wherein the controller is configured to generate the firstinformation packet with the first pattern information being indicativeof a first plurality of predetermined optical images spaced a firstdistance apart and wherein the controller is configured to generate thesecond information packet with the second pattern information beingindicative of a second plurality of predetermined optical images spaceda second distance apart.
 33. The computer input device of claim 31wherein the controller is configured to generate the first informationpacket with the first pattern information being indicative of a firstplurality of optical images spaced from one another by a distance whichvaries across the first optical pattern.
 34. The computer input systemof claim 1 wherein the surface comprises: a first surface portion on afirst side of a substrate and having the first predetermined patternportion disposed thereon; and a second surface portion on a second sideof the substrate and having the second predetermined pattern portiondisposed thereon.
 35. The computer input device of claim 7 wherein theoperating mode change comprises changing a font size based on thepredetermined image of an optical pattern then being detected.
 36. Acomputer input device providing input information to a computer,comprising: a position transducer configured to provide positioninformation based on movement of the computer input device relative to asurface, the position information for use in positioning a displayelement on a computer screen, the position transducer including apattern detector configured to detect substantially random imagepatterns on the surface and predetermined image patterns on the surface;and a controller coupled to the pattern detector, the controller beingconfigured to generate a message to the computer indicative of theposition information based on the substantially random image patternwhen the pattern detector is detecting the substantially random imagepattern and to switch to providing the position information based on thepredetermined image pattern when the pattern detector is detecting thepredetermined image pattern.